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Qingsi Wang, Mingyan Liu and Rahul Jain 



Abstract 

In this paper we introduce the problem of dynamic pricing of power for smart-grid networks. This is studied 
within a network utility maximization (NUM) framework in a deterministic setting with a single provider, multiple 
users and a finite horizon. The provider produces power or buys power in a (deterministic) spot market, and 
determines a dynamic price to charge the users. The users then adjust their demand in response to the time- 
■ varying prices. This is typically categorized as the demand response problem, and we study a progression of 

bJQi related models by focusing on two aspects: 1) the characterization of the structure of the optimal dynamic prices 

in the Smart Grid and the optimal demand and supply under various interaction with a spot market; 2) a greedy 
approach to facilitate the solution process of the aggregate NUM problem and the optimality gap between the 
greedy solution and the optimal one. 
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■ I. Introduction 

As the Smart Grid takes shape, new possibilities of efficient management of the electric power grid open 
q . up. One of these is pricing of electricity to consumers. Currently, temporal variations in the cost of electricity 
are hidden behind inflexible rate designs. This leads to inefficiencies with over-consumption during peak times, 
^ . and under-consumption during off-peak times. It also makes the problem of matching demand and supply, both 
O of which are uncertain, and affect grid stability, particularly acute. In the United States, only 60-100 hours of 
the year can account for 10-18 percent of the system peak load HI. Meeting this critical peak load requires 
installation, operation and maintenance of expensive combustion gas-turbine generators, since these start almost 
instantaneously. 
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CN . Dynamic pricing of electricity to consumers can remedy this problem by inducing consumers to switch off or 
defer some of the non-urgent, non-critical loads. For example, dish-washers in most households have a delayed 
start option. And yet, only a small fraction of consumers utilize this feature and use the appliance later at 
night at off-peak times. Dynamic pricing can make consumers sensitive to their time and amount of electricity 
consumption, thus potentially smoothing out peak-time system loads, and enhancing economic efficiency. 

Dynamic pricing rate designs are receiving increased attention by state commissions, with the California Public 
Utilities Commission (CPUC) having set a deadline of 2011 for the state electric utilities to propose dynamic 
pricing rate structures. These are defined as an electric rate structure that reflects the actual wholesale market 
conditions. Dynamic pricing can take various forms such as critical peak-pricing, time-of-use pricing and real-time 
pricing. CPUC defines the real-time price as the rate linked to the actual price in the wholesale hourly electricity 
market |2). Such prices can be communicated to the consumers over the advanced metering infrastructure (AMI), 
or a smart-grid network (3j and displayed on a smart meter. 

Literature Overview. Problems related to the design and operation of a Smart Grid are receiving increasing 
attention. The problem of dynamic pricing to shape demand is typically called demand response. There exists 
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a large literature on demand response; the most relevant is (H. This paper formulates the problem of optimal 
demand response in discrete time over a finite time horizon when users have some storage available, and the 
formulation can be categorized as a network utility maximization (NUM) problem (see O, and the references 
therein). Using Lagrangian duality, it establishes the existence of a solution, and gives a distributed algorithm 
based on gradient projection to compute this solution. The modeling of various appliances into the proposed 
optimization framework is also studied. In Q, a different approach is taken wherein demand is required to match 
supply, and users bid for load-shedding in an iterative supply function bidding mechanism, that was first proposed 
in flU. Both competitive and Nash equilibrium analysis is provided but the focus is on a single instant, and without 
storage. In J9), the focus is on optimal power flow through the network, wherein existence of a zero duality gap 
solution is proved. In contrast to aforementioned work, particularly H, in this paper we study a progression of 
related models by focusing on two aspects: 1) the characterization of the structure of the optimal dynamic prices 
in the Smart Grid and the optimal demand and supply under various interaction with a spot market; 2) a greedy 
approach to facilitate the solution process of the aggregate NUM and the optimality gap between the greedy 
solution and the optimal one. 

Other works have focused on electricity markets. While iTTOl provides a stability analysis of the wholesale 
electricity market, ifTTTl has focused on means for wind power generators to participate in the day-ahead electricity 
market. The difficulty in this is that wind-power generators cannot enter into binding contracts for a day-ahead 
power supply without assuming a huge risk. Thus, alternative market architectures are considered in fl2l . wherein 
entities called "aggregators" buy power from the wind power generators and participate in the day-ahead markets. 
The paper proposes optimal mechanisms that the aggregators can use to get the wind power generators to reveal 
their true distributions, and thus minimize the risk due to uncertainty in supply from the contracted wind power 
generators. A related paper is fl3l , wherein mechanisms for riskless dispatch by the generators are given for use 
with the current market architecture. 

Results and organization of this paper. The system model is first presented in Section|IIJ and three related problems 
are introduced. The first problem SYSTEM is a simplified special case of the main model with internalized 
production cost, which provides a preliminary framework for more realistic refinement. Its optimal solution is 
characterized in Section [III] based on a dual decomposition; moreover, a distributed algorithm to solve SYSTEM, 
which can be readily adapted for problems in the same family, is also proposed. In Section [TV] we extend the 
SYSTEM problem to the scenario where the provider may purchase in part from a spot market with an exogenous 
cost to narrow the demand-supply gap. This is referred to as the SYS_SPOT problem, and the interaction between 
the decision of purchase and the spot market price is investigated. We then consider the explicit effect of the 
production cost in a third problem in Section [V] and the structure of its optimal control in the single-user scenario 
is studied. A greedy algorithm to approximate the solution to SYSTEM is presented in Section|VTJ where an upper 
bound on the optimality gap is established and heuristics on evaluating this gap are discussed. Numerical results 
of sample problems are illustrated in Section |ViTl We present an attempt of adopting distributed Newton's method 
to problems in this work in Section IVIIII and Section |IX] concludes the paper. 

II. System Model and Notation 

Consider a smart-grid network that consists of n users or households, and one provider. We consider a discrete- 
time model with a finite horizon tf. A user i consumes Xi(t) power at time step t, and the supply by the grid 
is denoted as zi{t). The excess supply /demand is charged to/discharged from the user's battery with capacity 
Bi. The energy level/state of the battery before operation at time t is denoted by yi{t). We assume that the 
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total supply from the grid at time t is upper bounded by Ct, and the total demand of user i over time is upper 
bounded by Di. For convenience we define the following vector forms of the above quantities: the demand 



of user i, Xi 



Xi(0), Xi(l), 



the state of user €s battery, y 



Xi(t f ) ; the supply of the grid Zi = z,(0), ^(1), 

i T 



• (*/) 



; the battery capacities of users B 



of users D 



; the supply constraint of the grid C 
D u D 2 , ••• , D v 



Co, Ci, 



c. 



T 



; the demand constraint 



. Also, x = vect(xi, . . . , x n ) with vect(iti, . . . ,u n ) being the stacked 
vector from u\ to u n , and similarly define y and z. When user i draws power X{ over time, her satisfaction is 
measured by the utility function 



Ui(x t ) = Y J V i (x t (t)), 



t=o 

where V- L : R + — > R + is a strictly increasing, strictly concave and twice differentiable function of Xi(t). Denote 
by yi(t) the change of battery state of user i at time t: 

Vi{t) = y%(t + 1) - yi(t) = Zi(t) - Xi(t). 

Assuming zero initial state, y(0) = 0, we then have 

t-i t-i 
Vi(t) = Vi(0) + ^2 m(s) = ^2(zi(s) - Xi(s)). 

Therefore, 



s=0 



s=0 



Vi = H(zi - Xi 



where 



H 



1 

1 1 

1 1 

1 1 







1 

1 1 



G r(*/+i)x(*/+i). 



In addition, since the battery suffers wear and tear due to usage, we model maintenance and operational costs as 

tf+i 

M{yi) = £ m(i/i(i)), 
t=i 

where m : R + — > R + is a strictly increasing, strictly convex and twice differentiable function of yi(t). The 
objective is to maximize the sum of individual utilities of both users and the provider or the social welfare. 
Formally, we consider the following related problems in this paper. The main model is given as follows. 



maximize ^ (Ui(xi) - M(yi)) - P(z) 

i=l 

subject to yi < Sjl 

Hi = H(zi - x^ 



(1) 

(2) 
(3) 
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J>=<C (4) 

i=l 

< A (5) 

Xi,yi,Zi yQ,i = l,2,...,n (6) 

where P : R™(* /+1 ) _ >. R + i s the production cost function to be specified later, and 1 = (1,1,..., 1) T with 
proper sizes. We begin with a simplified special case of this problem by assuming P = 0; this is denoted as 
SYSTEM (U,M,B,H,C,D). The motivation for studying this degenerated model is twofold: 1) it models the 
scenario when the production cost is realized thus fixed before the distribution of energy or when the marginal 
cost is zero such as in the case of renewable energies; 2) more critically, it provides a preliminary framework that 
reveals the basic structure of optimal solutions and their algorithmic implementation that can be readily extended 
to more realistic models. 

We then consider the setting wherein the grid may purchase power from an external spot market to narrow the 
demand-supply gap, while incurring an exogenous cost; this is formulated as the SYS_SPOT (U, M, B, H, C, D; a) 
problem as follows. 



maximize ^ (Ui(xi) — M(yij) — a T 

i=l 



n 



5>-c 



,i=l 



(7) 



subject to yi < Bil (8) 

Ui = H(zi - Xi) (9) 

l T Xi < Di (10) 

Xi,yi,Zi h 0,i = 1,2, . . . ,n (11) 



where a 



CO, 0*1, ••• °~t f 



is the price vector of unit power in the spot market, and [•]+ = max{-, 0}. Our 
main interest in this problem is the characterization of spot market prices that may incentivize the exogenous 
purchase. In particular, we show that there is a threshold price below which this purchase is justified, and it is 
closely related to the optimal solution to SYSTEM. 

Our last model considers non-trivial production costs. We consider the linear cost in the original setting with 
multiple users, that is, P(z) = cr T d z , where d z = Ylt=i z i i s tne vect or of aggregate demands, and a can be either 
interpreted as the unit power generation cost or the spot market price when the provider produces nothing of its 
own and completely resorts to the spot market for supply. This model is denoted by SYS2 (U, M, B, H, C, D). 
We also consider a general convex cost in the single-user case that has an explicit control structure. 

In the next section we start with the SYSTEM problem, and note that when constraint (0]) is decoupled, the 
SYSTEM problem can be separated into several subproblems. This observation motivates us to consider the 
dual-decomposition based technique to solve the SYSTEM problem Q, 0, which we show next. 

III. Dual Decomposition of system and the Properties of Its Solutions 

In this section the SYSTEM problem is analyzed based on a dual decomposition, which also prompts a distributed 
first-order algorithm to solve it, and the properties of both the primal and the dual optimal solutions (demand, 
supply and prices) are discussed. 
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A. Dual decomposition 

We first reformulate SYSTEM (U, M, B, H, C, D) as 



maximize ^ (Ufa) - M( yi )) (12) 



i=i 



subject to vect(x, y, z) € T (13) 
over vect(x, y,z) ET> (14) 



where the domain and the feasible region are respectively given by 

V = Jvect(x, y, z) 

and 



Xi h 0,l T Xi < A, 
O^Vi^ Bil,Zi h 0,Vt 



7" = |vect(x,y,z) j/j = - X;),y^^ ^ C,Vi| . 

Note that relint Dfl J / when B, C and D are all positive, as we shall assume so. Indeed, let e > be 
chosen later, and consider 

xAt) = a , 

w tf + 1 

where < a < 1 satisfying 

V" D 

a l ~ l < mm C t , 

t f + l 

Zi(t) = Xi(t) + and y~i(t) = 5Zs=o(^( s ) ~~ %i( s ))- Let e < mmj{min{.Bj, Z^}}, and it can be verified that 
vect(x,y, z) £ relint 2? n J 7 . In addition, since this problem is convex and the inequality constraint is affine, 
Slater's condition is then satisfied, and hence, strong duality will hold. Because of the strict concavity of the 
objective function in x and y, there are unique optima for x, y and z. As pointed out before, the structure of the 
SYSTEM problem prompts a solution based on a dual decomposition. Its Lagrangian is given by 



L = (Ufa) - M( yi )) + vT{H(zi - Xl ) - Vi ) + 9 T (C - ]T *) 

i=l i=l i=l 

n n n 

= [Ufa) - ufHxi) - £ (Mfo/i) + uj yi ) - - H T Ui) T Zi + e T c. 



1=1 1=1 1=1 

Let v = vect(z/i, . . . , v n ). The dual function is then given by 

g(v,9) = max L(x,y, z,u,9) 

vect (x,y,z)£T> 



^ max Wj(ji; ^) - y~) q ^mir^ ^Rjjyi; Vj) - y~) ~ H T Vj) T Zi + 6 T C, 



i=l l T Xi<D, i=1 



where 



and 



Wfa; Vi) = Ufa) - vjHxi, 
RiiVu Vi) = M(yi) + vfyi, 
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and we introduce 

USERi : maximize Wj(xj;i/j) 

subject to Xi >z 0, 1 Xi < Di 

and 

USER^ : minimize Ri(yf,Ui) 

(16) 

subject to < yi < B{1 

as two subproblems for user i. Since the minimization with respect to Zi is a linear problem, which is readily 
solved, the dual problem is given by DUAL_SYS (U, M, B, H, C, D): 

n 

minimize ^ (W*(vi) - R*{vi)) + 9 T C 
i=i 

subject to 9 - H T Vi y 0, 9 y 0, i = 1, 2, . . . , n 

where W*{i>i) and R*{vi) are the optimal values of problem ( fT51 ) and (fT6l ) for a given fj. Given now this dual 
decomposition structure, a distributive algorithm can be readily proposed to solve SYSTEM, and this is widely 
studied in the literature, see e.g. [0, |@). We report our version of the algorithm in Appendix lAl and show its 
convergence. 



B. Properties of solutions to SYSTEM and DUAL_SYS 

Denote by x* = vect(x^, . . . , x*) the optimum for x in SYSTEM; v* is an optimal solution for v of DUAL_SYS, 
and similarly define y*, z* and 9*, respectively. We first note that H T Vi can be interpreted as the price vector 
that the provider charges from user i for each unit of consumed energy, in light of subproblem USER^, given that 
it is component-wise non-negative. Indeed, the following result establishes this premise. 

Proposition 1: H T v* y for all i. 

Proof: The proof can be found in Appendix |B] ■ 

Remark 1: Since Zi(t) is the total amount of energy distributed to user i at time t, under the interpretation of 
H T v i being a price vector it is as if the energy buffered in battery were only charged when it is later consumed 
while incurring a storage cost before the consumption. This obviously is not what actually happens in practice, 
but serves as an interesting interpretation of the result. We do note that if there is no battery buffering in the 
model, this interpretation is consistent with the notion of shadow price in the literature (see e.g. lfT4l . |[T5l ), i.e., 
entries of H T Ui are shadow prices of power. 

Hence, 6*(t) = maxj (i^ T i/*) t for all t. Moreover, 1) if we assume that x\ G int Xi for all i, where X; t is 
the feasible set of ( fT5T ), i.e., Xi = {xi \ Xi y 0, l T Xi < D^, H T v* is then uniquely determined for each i; 
2) alternatively, if we assume that x* € int Xi for at least one i, say i^, and zf y 0, H T i/f is then uniquely 
determined and from the minimization with respect to Zi k we get that 9* = H T v* k . In either case, 9* is also 
uniquely determined. 

In Section [IVJ we will consider the situation where there is a gap between the users demand and the power 
generation capacity of the grid at some time instants. At those instants, the grid may purchase power from the 
spot market to narrow the demand-supply gap. We will show therein that the above unique 9* is a "lower bound" 
on the price of unit power in the spot market above which the grid decides to make no purchase. 
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Also, we can determine the closed form of y* given v* based on (fT6l) . Since the objective and the constraint 
are both separable in time, we conclude 

yt(t) = [m'- 1 (-ut(t-l))}** 

for all i and t > 0, where is the projection operator onto [a, b], i.e., [x\ h a := min{max{x, a}, b}. Let 
p\ := H T v*, and p*(t) := p\(t) - p*(t - 1) = - 1). We then obtain 

yt{t) = [w!- 1 mt))]o i - 

If we assume that < y*(t) < B, L , the above result can be rewritten as 

Ptit) = m'(yUt)). 

Therefore, at optimality the marginal cost of reallocating the power purchase from t — 1 to t is equal to the 
marginal cost of storing the energy in battery after purchasing in t — 1, which is intuitively appealing. When 
the demand is known and fixed a priori, some structural results on the optimal power generation and battery 
scheduling in a single user setting can be found in |[T6l . In Section IVl we will also discuss in detail the explicit 
structure of optimal control in the single-user problem when a non-zero production cost is incorporated in our 
main model. 



IV. Spot Market 



In this section, we consider the SYS_SPOT problem. As before, we reformulate this problem and its domain 
and the feasible region are the same as in the SYSTEM problem. Also, there are unique optima for x, y and z 
because of the strict concavity of the objective function in x and y. 

Denote by z*(a) = (z*(a), . . . , z^(a)) the optimal solution for z of SYS_SPOT given a. Define 



a 



In the following, we characterize S using the solutions of the SYSTEM problem and its dual problem DUAL_SYS. 

To avoid ambiguity, symbols without hat are variables in SYS SPOT, and others are for the SYSTEM problem 

and its dual. We assume that either of the two cases below is true: 

Assumption 1: 

a) x* € int Xi for all i, or 

b) x* € int X for at least one i, say ik, and z* >- 0. 

Recall that x* denotes the optimal solution to the SYS_SPOT problem for Xi, and Xi is the feasible set of ( 031 ). 
In the following proposition, we show that S can be characterized with a threshold of prices. 

Proposition 2: a € S if and only if a y 6*, noting that 9* is the optimal dual variable in DUAL_SYS 
(U,M,B,H, C,D). 

Proof: 1) Necessity. First of all, the Lagrangian of SYS_SPOT is given by L(x, y, z, v, p), 



L = Y J (U l {x i )-M{y i ))-a 1 



n 



E 

.i=i 



Zi 



C 



+ Y j yJ{H(z l 



n 

i=i 



Zi 



C 



Vi) 

n 

+ ufHzi 



i=i 



8 



minimize a T 



Group the terms in a similar way as in DUAL_SYS, and the dual problem is 

n 

minimize J] (Wf (n) - - °) 

i=i 

subject to /Uj + f j ^ 

Mi ^ 0,i = 1,2, . . . ,n 

where W* and -R* are the respective optimal values of problems (fT5T ) and (fT6l ) for a given i>i as in DUAL_SYS, 
and G*{v; a) is the optimal value of 

- + 

subject to Zi y 0, i = 1, 2, . . . , n 

Because of strong dualitjQ, the optimal solution (x*,y*,z*) to SYS_SPOT is also the maximizer of the 
Lagrangian L(x,y, z,u*) where v* is an optimal solution of the dual. Therefore, z* is an optimal solution of 
(fTTT ) when v = v*. Since Y^i=i z t * s nn i te > we must have a y H T v* for all i. On the other hand, provided 
Ya=i z t ^ C> we can obtain the same solution (a;*, y* , z*) of SYS_SPOT by solving the SYSTEM problem with 
the same parameter set (U,M,B,H,C,D). Recalling the properties of solutions to SYSTEM in Section UlI-BI 
under assumption a) we have u* and v* are uniquely determined and u* = V* for all i. In addition, 0*(t) = 
maxj (i? T P^) for all t. Under assumption b) we have u* k = and 0* = H T u* k . In either case, it follows that 
a y 0*. 

2) Sufficiency. We only provide an outline of the proof here. Our strategy is to show that the optimal solution 
(x*,y*,z*) to the SYSTEM problem with optimal dual variables (z/*,#*) also satisfies the KKT conditions of 
SYS_SPOT when a >z and then because of strong duality and convexity of SYS_SPOT, it is also the optimal 
solution of SYS_SPOT with optimal dual variables u* = u*, given the spot market price a. To this end, note that 
the KKT conditions of SYS_SPOT only differ from those of SYSTEM in the dual feasibility and the stationarity 
conditions because of the replacement of 9 with a and the non-differentiable term in the objective function. Recall 
that 0*(t) = maxj (iJ T i^) t for all t, and we have a y H T u* if a y 9*. Let d Zi L denote the subdifferential 
of the Lagrangian of SYS_SPOT with respect to z% for all i. Hence, when a y 9* and U{ = v\, G d Zt L for 
all zi = z*, and hence combining other optimality conditions, we obtain (x*, y*, z*, v*) is a primal-dual optimal 
solution given a y 9*. ■ 



V. SYS2 AND THE STRUCTURE OF OPTIMAL CONTROL FOR A SINGLE USER 

Under some circumstances, we have an explicit form of the solution to the SYS 2 problem, which is given by 
the following proposition. 

Proposition 3: At time t, £)" =1 z*(t) = C t if (H T u*) t > a t for some i, and z*(t) = if (H T u*) t < o t . 
Proof: The results follow from the KKT conditions. Firstly, the dual feasibility and the complementary 
slackness specify that 

n 

9 y 0,9 T (C -J2 z i) =0 - 

1=1 

'it can be shown that the reformulated SYS_SPOT problem is convex and Slater's condition is satisfied. 
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Furthermore, consider the Lagrangian of SYS 2 and by the stationarity condition we have 

a + 9- H T ^ y 0, (cr + 6 - H T ^) T Zi = 0. 

for all i. Thus, if {B. T Vi) t > ft for some i at time t, then 9(t) > 0. Therefore, 5^" =1 Zi(t) = Ct- On the other 
hand, if (H T Vi) t < at, we have at + Q{t) — {H T Vi) t > 0, and hence Zi(t) = 0. ■ 
In the rest of this section, we focus on the decision-making by a single user and a single provider, and derive 
the structure of optimal policy. We relax our previous constraints such that they are now separable in time, and 
we assume that y(t + 1) € [0,5], x(t) € [0,5] and z(t) € [0, C] for all t = 0, • • • , tt, where the user index 
i is now omitted. We assume that if a quantity z(t) is produced at time t, the cost is P t (z(t)) where Pt is 
strictly increasing and convex in z(t). Most of our notation is consistent with previous sections except that M 
will signify the scalar maintenance cost. Then, the decision-making problem is given by the following optimal 
control problem, OPT_CONTROL: 

max Y?tU[V^))-Pt{z(t))-M{y(t + l))\ (18) 
s.t. y(t) = z{t) - x(t),Vt }p(t) 

Ho(t){ 0<y(t)<B,\ft, 
X (t){ < x(t) < D, Vt, }Ai(t) 

6 (t){ < z{t) < C, Vt, }#i(t) 

2/(0) = yo, 

where y(t) = y(t+l)—y(t). Denote the dual variables to the above constraints by p(t), /jio(t), fJ-i(t), Xo(t), Ai(t), Bo(t) 
and 6\(t), as shown above. Then, due to strong duality, the KKT conditions suggest that there exist y*, x*, z* , 
p*, Pq, p\, Aq, A^, 6*q, 6\ that satisfy 

V'{x\t)) - p*(t) - \$(t) + A$(t) = 0, (19) 

-p t V(*)) + P *(t) - et(t) + eut) = o, (20) 

-M'(j/*(t)) + p*(t) - p\{t) + (4(t) = 0, (21) 

(4(t)y*(t) = Q, pt(t)[B-y*(t)]=Q, (22) 

A$(t)s*(t) = 0, AJ(t)p - x*(t)] = 0, (23) 

flg(t)z*(t) = 0, 9l(t)[C-z*(t)]=Q, (24) 

such that (z*, z*) is the optimal control law and y* is the state of the system, where p(t) = p(t) — p(t — 1) for 
t > 1. 

Now, from KKT conditions (|2TI ). we have 

t t 

p*(t) = M'(y*(s)) + $>i(s) - /iS(*)] + p*(0). (25) 

8=1 S=l 

Further, from KKT conditions (l20l ), we have 
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for all t. From this, using the complementary slackness conditions d24l ). we can conclude that either Pt(0) = 
p*(t) + 6ft (i), or P t (C) = p*(t) - 0\(t), or P t (z*(t)) = p*(t). We can summarize this as 

z\t)= [iTV(*))]?- (26) 
Similarly, from KKT conditions ( fT9l ), we get 

= p*(t) + (XUt) - AS(t)), 
which together with the complementary slackness conditions d23l ), implies that 

x*(t)= [V-V(t))]£. (27) 

From ( f26T > and d27l ), we conclude that is non-decreasing and x*(t) is non-increasing in p*(t) for each 

t. Moreover, x*(t + 1) < x*(t) whenever p*(t + 1) > p*(t). In the following, we assume that P t = P, and 
consequently z*(t + 1) > z*(t) whenever p*(£ + 1) > p*(t). Using (|25T ) and the complementary slackness 
conditions (1221 . we also conclude that if y(t) > for all t € T = {ti,ti + 1, . . . ,T2}, p(t) is then strictly 
increasing in t over T. 

Combing the results obtained above, we conclude the following only possible dynamics of the optimal control, 
and the detailed analysis can be found in Appendix |D] 

(i) x*(0) > z*(0) when yo > 0, and x* is non-increasing, y* decreasing, z* non-decreasing in t. Either 
t x=z = t y= o < tf (i.e., x*(t) = z*(t) and y*(t) = simultaneously) and from then on the control and the 
state maintain fixed until t = tf, or x*(t) > z*(t) and y*(t) > until the time horizon. 

(ii) x*(0) = z*(0) when yo = 0, and the control and the state maintain fixed until t = tt. 

In addition, once x*(t) = z*(t) = u, we have u = [min{C, D, u}] + where u is such that V'(u) = P'{u). 

To conclude, we sketch the dynamics of the optimal control and the state of case (i) in Figure [T] Note that the 
change of x*, y* and z* are not necessarily linear. 



X, z 




+ 



ty =Q tf 



Fig. 1. The optimal policy for a single user 

VI. A Greedy Approach 

In this section, we consider a greedy approach to approximating the solution to the SYSTEM problem. The 
need for a greedy approach can be motivated by the following considerations: 

1) depending on the definition of time scale of a step, the convergence time may exceed the time step; 
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2) some parameters such as B, C and D may be stochastic within the time horizon, and we can only observe 
their realizations a posteriori. 
Thus, we consider the following sequence of single-step optimization problems as approximations to the SYSTEM 
problem. We denote each by SYS_GREEDY (V, B,C t , D;t), 

n 

maximize ^~]Vi(xj (t)) (28) 

i=i 
n 

subject to ^Xj(t) < C t (29) 

i=i 

t-i 

Xi(t) + J>*(s) < A (3°) 

s=0 

Xi(t) > 0,i = 1,2, (31) 

where (x*(s),i = 1, . . . , n) is the optimal solution to SYS_GREEDY (V, B, C s , £>; s). Note that because of the 
myopic nature of each single-step optimization, it does not consider utilizing battery to store energy for future 
use, and thus the dimension of the greedy formulation reduces to that of the space of x. 

There are a number of interesting problems to investigate. One of them is whether there is a speed advantage of 
SYS_GREEDY over SYSTEM, and the other is the optimality gap between them. We will present numerical results 
on the convergence rate of SYS_GREEDY in Section IVTIl and in the rest of this section we will characterize the 
solution of SYS_GREEDY and subsequently the optimality gap. 

In the following presentation including Section IVIH we limit our discussion to the case where the utility 
functions of users at a given time step are identical, i.e., V\ = V. We denote by x\ and xf'* the optimal solutions 
of SYSTEM and SYS_GREEDY, respectively. Define x*' 9 = l T;c f* and X* = 1 T £*, and denote by %, W and 
J\f\ the output of the following procedure. 

Procedure GetChi(B, C, D) 

Initialize jV 4- {1, 2, . . . , n}, M\ <- 0,prev_7Vo <- No 
repeat 

prev_AA i- No 

for i G No do 

if (l T C - £. eM Dj)/\N \ < A then 

Xi^(i T c-E j6M A0/l^o| 

else 

Xi <- A 

No 4- No- {»}, M<-Mu {i} 
end if 
end for 

until No = prev_A/o or No 4- 
return x i} i = 1, . . . ,n,No,Ni 

The above procedure can be literally described as a best-effort balanced water-filling process. That is, it attempts 
to fill each water tank (user) the same amount of water (energy), subject to their capacity (demand) constraints, 
and whenever smaller ones of them overflow, the excess goes to larger tanks in equal amounts. The following 
proposition asserts that this procedure in fact results in the optimal total per-user amount of energy. 
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Proposition 4: x*' 9 = X$ = Xi f° r au *■ 

Proof: That x*' 9 = Xi f° r ai l * * s straightforward, and we prove that x$ = Xi f° r au * i n Appendix ■ 

Remark 2: The balanced property of optimal solutions is a consequence of the concavity and the user- 
independence of utility function that we assumed. For user-dependent utility functions, we conjecture that the 
solution may be in a scaled form of balanced power distribution. 

Let p* and p 9 '* be the optimal value of SYSTEM and the sum of optimal values of SYS_GREEDY, respectively. 
Note that Xi(t) = %t (t%) >* = ^,---,n,t = 0,...tf, are feasible solutions to the sequence of greedy 
optimization. The optimality gap is then upper bounded by 

n t f n „ 

i=l * t=0 i=X 

Eqn. (l32l ) suggests that the optimality gap would shrink as the power generation capacity C is more balanced 
among time steps. Moreover, observe that the loss of optimality of the greedy approach results from the myopic 
optimization that overlooks the use of energy buffer (i.e., battery) to balance energy dispatch between steps. We 
thus conjecture that the vector C in a roughly ascending order could yield a smaller optimality gap. 

VII. Numerical Illustration 

In this section, we illustrate the demand sensitivity to spot market price that we studied in Section JV] using 
two sample problems that are specified in Table J] We also empirically show the advantage in the convergence 
rate of the proposed greedy approach compared to the SYSTEM problem, when the respective optimization is 
performed using the same distributed algorithm based on gradient projection. 




(a) <t(0) = k ■ 0*(O) (b) cr(2) = k ■ 0*(2) (c) a = « ■ 9* 



Fig. 2. Demand sensitivity to spot market price for the sample problem n = 10, tf — 4. 




(a) o-(O) = k ■ 0*(O) (b) cr(3) = k ■ 9*(3) (c) a = k ■ 0* 



Fig. 3. Demand sensitivity to spot market price for the sample problem n = 20, tf = 6. 

As shown in Proposition |2j 9* assumes the threshold value of spot market price a, and in Figure |2] and [3] 
we tune a in three different ways to demonstrate the behaviors of optimal demand. The tests are performed 
for both sample problems and we elaborate the one with n = 10, = 4 for example. In the first scenario, we 
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set a(t) = 0*(t) for t > and er(0) = k ■ 9*(0), where k is a tunable parameter. We scale the third entry of 
a in the second scenario and the entire price vector in the third one. In Figure |2] and [3j we plot the curves of 
II EiLi z i ~ C} + ||i versus k, respectively. As can be seen in the figures, the demand increases as k decreases when 
k < 1 until hitting the demand constraint, and remains unchanged when k > 1, as suggested by Proposition |2] 



TABLE I 

Parameters of the two sample problems in Section[VTIJ 





B 


C 


D 


n = \Q,t f = 4 


[10,...,10] T 


[8,7,...,4] r 


[1,2, ...,10] T 


n = 20, tf = 6 


[10,...,10] T 


[10,9, . . . ,4] T 


[0.5,1,...,10] T 


V(a) = log(l + a) and m(b) = ±b 2 



TABLE II 

Total runtime (iterations) of sample problems. 





SYSTEM 


SYS_GREEDY 


n = 10, t f = 4 


25.0959s(100) 


0.0206s(210) 


n = 20, t f = 6 


68.0778s(158) 


0.0316s(197) 



In Table OH we report the respective convergence rates of SYSTEM and SYS_GREEDY. Though these records 
are machine-dependent, it demonstrates the significant comparative advantage of the latter in speed, and the 
greedy approach can be used as a good approximation to SYSTEM when the optimality gap bounded by d32l is 
acceptable. 

VIII. A Distributed Primal-Dual Algorithm: Newton's Method 

In this section, we present a distributed second-order algorithm, namely Newton's method, to solve the SYSTEM 
problem. We first reformulate SYSTEM and approximate it with an equality-constrained problem, and apply 
Newton's method on the primal problem. Theory and application of distributed Newton's method in network 
optimization can be found in ifTTl . lfl"8l . lfl9l and etc. Instead of solving the dual problem where updating 
Lagrange dual variables can be interpreted as dynamic pricing, this algorithm works on the primal side and 
the solving process can be regarded as dynamic resource slicing [HI. In this section, we assume that —U{ is 
self-concordant EUl for all i for the proof of convergence. A convex function / : R —¥ R is self-concordant if 
\f"'{x)\ < 2f"(x) 3 / 2 for all x € dora/, and self-concordant functions include linear, convex quadratic functions, 
negative logarithm and etc. 

We first introduce non-negative slack variables to three inequality constraints, that is, 

n 

l T Xi+pi = Di, yi + qi = Bil, ^ Zj + r = C, 

i=l 

and using the logarithmic barrier function for non-negative constraints, we approximate our original problem with 
an equality constrained problem as follows. Let p = vect(p 1; . . . ,p n ) and q = vect(gi, . . . , q n ). To simplify our 
notation, we also define w = vect(x, y, z,p, q, r), that is, for instance, Wi = Xi for 1 < i < n and wi{t) = Xi(t) 
and etc. We also define the logarithm for vector u = (u\, . . . , u m ) 6 R m to be 

iogu = y~]iog ut. 

t 
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Let 

n In 5n+l 

i=l i=n+l i=l 

and / is also self-concordant for r > 1. Define 



A = 











In 











^n(t/+l) 













r 


In(t f +1) 


-r 

















s 








!t f +l 



where I(m) is the m x m identity matrix, 











H 








1 T 


,r = 




H 



and 5 = [i^+i, . . . , h s +i\, and also define b = vect (D, B\l, . . . , S n l, 0, C). The original problem can be then 
approximated as 

minimize f(w) 
subject to Aw = b 

It can be shown that as r approaches zero, the optimum of the above approximating problem converges to that 
of the original one. 

Like the gradient projection method, the Newton's method solves the above problem through an iterated update 
procedure. In each iteration, we have 

w k+1 = w k + r y m a k Aw k , 

where a k is the step-size, Aw k is the Newton direction, 7 is the discounting factor and 

m = min{m' G Z + | w k + -y m 'a k Aw k >z 0}. 

Let H k = \7 2 f(w k ) and = V/(x fc ). The Newton direction is then given by the solution to the following KKT 
system, 





A T ~ 




Aw k 




9k 


A 







v k 








where v k are dual variables associated with the equality constraint and is null matrices with proper dimensions, 
and it yields 

Aw k = -H-\g k + A T v k ), 
(AH^A^v' = -AH~ x g k . 
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Note that / is separable for each Wi(t), and hence gu only depends on local information with terms 



Of 



dwi(t) 



-*?(**(*))- ife. 


iOi(i) 


= Xi(t) 




Wi(t) 


= m(t) 




Wi(t) 


= u{t), 



u = Zi,pi,qt,r 



and Hk is a diagonal matrix with terms on the diagonal 



d 2 f 



-V"i{Xi(t)) + j^jy, U^i) = 

m"i(yi(t)) + toj(t) = yj(t) 

= it(i), 



u(t) 



u = Zi,pi,qi,r 



Therefore, Aw k can be computed from collected data distributedly given the vector v k . Define the Newton 

\(w k ) = {Aw T k H k Aw k ) l l 2 , 



decrement X(w k ) as 
and use the step-size lfl"8l 



k 

a = < 



A0^+I> ifA(x fc )>i 



1, o.w. 

where c is some positive scalar that satisfies 5/6 < c < 1. The description of algorithm is then as follows. 

Second-Order Algorithm for Power Distribution 
User i's algorithm At iterations k = 1, 2, . . ., 

1) Receives from the grid Aw k and \(w k ), and updates a k and w k+1 . 

2) Computes the associated terms in g^+i and fljt+i, and communicates them to the grid. 
Grid's algorithm At iterations A; = 1, 2, . . ., 

1) Receives from each user and ii^. 

2) Computes w fe , Au;^ and X(w k ), and broadcast them to all users. 

Note that our implementation is different from that in lfI71 and |[T8l . where the computation of v k is centralized 
at the grid, due to our introduction of partial computational capacity of the grid. The self-concordance based 
convergence analysis is similar to [18 ] where r is assumed greater than 1 to preserve the self-concordance of /, 
and the analysis is omitted for brevity^. 

IX. Concluding Remarks 

In this paper, we have studied key features of optimal dynamic prices in smart-grid networks. Moreover, an 
attempt of adopting distributed Newton's method to the model in this paper, namely the SYSTEM problem, is also 
presented, which is however mainly designed to solve the primal problem and is categorized in the domain of 
dynamic resource slicing instead of dynamic pricing. In this work, the explicit control structure for time-invariant 
power production and energy buffering costs has been studied, and continuation of this effort under more general 



2 For the simplicity of presentation, we implicitly assume that the error in computing v k is negligible. In 1181 . the authors assume the 
Hk -quadratic norm of error is upper bounded. 
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cost functions is one direction of our future research. On the other hand, deviating from the deterministic setting 
and investigating the optimal pricing with stochastic demand and supply is also one ongoing research project. 

Appendix A 

A Dual-Decomposition Based Distributed Algorithm 

Let 

n 

q(u, e) = J2 i w i^) ~ R ^)) + 9T °> 

i=l 

and 

£ = !vecb(p,v,0) 9 - H T ^ y 0,(9 t 0,Vi} . 

Define x*(ui) and as the respective unique maximizers of (fl3T ) and (fl6l ). Let Q k = Q(u k , k ). We solve 

DUAL_SYS using the gradient projection method, and the updating rule in the fcth iteration is given by 



V k+l = v\ - a k V^Q k = u k + oF{Hx*M) + V*("i 

Qk+l = Q k _ a k Ve Qk = Qk _ a k C 

and 



vect(v k+1 ,9 k+1 ) = vect(? 



-k+l nk+l\ 



where a k is a diminishing step-size satisfying 



oc 



J2 ak = °°^(« fc ) 2 < °°' 

k=l k=l 

e.g., a k = with T a fixed non-negative number, and [xq]x is the projection of xq onto a set X , or defined 
as the following constrained quadratic program 

l \\ 112 

minimize —\\x — xq\\ 2 
subject to x € X 

Because all gradients are bounded with diminishing step-size a k , the gradient projection method converges 
for the dual problem ETi . |[22l . fl5j . Since x*(ui) and y*{vi) are the respective unique maximizers of (fT3T ) and 
(fT6l ) and the strong duality holds, x*(v*) and y*(v*) are primal optimal, where v* are the solutions of the dual 
problem. To summarize, we have the following: 

First-Order Algorithm for Power Distribution 
User i's algorithm At iterations k = 1,2, . . ., 

1) Receives from the grid the dual variables v\ 

2) Computes x*(u k ) and y*(y k ), and communicates them to the grid 
Grid's algorithm At iterations k = 1,2, . . ., 

1) Receives from each user x*(u k ) and y*(y k ) for all i 

2) Computes the dual variable v k+1 and 6 k+1 for all i, and communicates them accordingly to each user 
3 With a slight abuse of notation, we will use x(t) to denote the tth entry of x, and x{y) as a function of v. 
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The algorithm would terminate when the following stopping criterion is satisfied: 

\\vect(u k ,9 k ) - vect^- 1 ,^- 1 )!! < e, 

where e is a predefined constant. 

Appendix B 
Proof of Proposition [TJ 

We prove by contradiction. Assume that there exists at least one i such that {H T v*) t = Y?JLt v i( a ) < for 
some t. Let / = aigmm t (H T v*) t . We first note that x* ^ and further more there exists to G / such that 
x*(to) > 0. Otherwise, one can switch the value of x*(to) with that of some positive entry in x* and obtain 
an improved value of Wi(xnuf), which is a contradiction to the fact that x\ is the maximizer of Wi{x,i\vf). 
Therefore, 

1) if there exists t < t such that (H T uf) t > (H T uf) to , let t x = max{t : {H T v*) t > (H T v*) tol t < t } and 
we then have u*(ti) > 0; otherwise (H T u*) tl < (H T u*) to . Moreover, u*(t) = for t\ < t < t if any. 
We then set 

"!(to) *- ft (to) + e 

and 

"*(*i) " e 

where < e < min{f*(ti), —(H T v*)t }, where "y ^— x" signifies assigning the value of x to y. By doing 
so, the value of Wi is strictly decreased while that of Ri is non-decreasing and the feasibility constraints 
are maintained. We thus obtain an improved dual objective value, which leads to a contradiction. 

2) If {H T v*) t = {H T v*) to for all t < t if any, we set 

"i(to)<-ut(to) + e 
where < e < —(H T h>*) to , and the same argument applies as in 1). 

Appendix C 
Proof of Proposition [4] 

Note first that ^27=1 ~Xa, = Ya=i Xi = min{l T C, l T D}. The proof of this claim is straightforward and we 
thus omit it here for brevity. Without loss of generality, we assume that D is sorted in an ascending order, 
and therefore \% i s non-decreasing in i as the procedure suggests. Moreover, if \j < ~X% f° r some j < i, then 
Xj = Dj. Assume for contradiction that x* < Xi < f° r some i, and there then exists at least one j ^ i such 
that Xj > Xj- Thus, 

1) if j < i, then Xj < Xi- If Xj < Xi> we ^ ave Xj > Xj = Dj, which is infeasible. Therefore, Xj = Xi 
and consequently Xj > Xi- Hence, there exists some t such that x*(t) > x*(t). Consider an e such that 
< e < min{A - X*, (x*(t) - x*(t))/2} and set x*(t) <r- x*(t) - e and x*(t) <- x*(t) + e. We then 
obtain a feasible but improved objective value, which is a contradiction. 

2) If j > i, then \j ^ Xi- Hence Xj > Xi an d the same argument applies. 

The above argument also applies symmetrically to the case when Xi < Xi < Di for some i. Therefore, x\ = Xi 
for all z. 
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Appendix D 

Dynamics of the Single-user Optimal Control 

We rule out the suboptimal cases as follows. 

1. If x*(Q) < z*(0), then y*(l) > and p*(l) = p*(0) + M'(y*(l)) + /U*(l) > p*(0). Hence, z*(l) > z*(0) and 

< x*(0), and the battery keeps charging until either fully charged before the time horizon and maintain 
the state, or the time horizon is reached; however, this is clearly suboptimal. 

2. If x*(0) > z*(0), which is only feasible when yo > 0, let us consider the sequence of events x*(t) < z*(t) 
and y*(t) = 0. Let t x < z be the first time step when x*(t) < z*(t) or be infinity if it does not happen before the 
horizon, and similarly we define other quantities. 

2a) If x*(t) < z*(t) happens before y*(t) = 0, i.e, t := t x < z < t y= o, then y*(t + 1) > y*(t) > and thus 
p*(t + 1) = p*(t) + M'(y*(t+ 1)) + fi*(t + 1) > p*(t). Hence, z*(t + 1) > z*(t) and x*(t + 1) < x*(t), 
and again we are in case 1, which is suboptimal. For the boundary case that t x<z =tf, the suboptimality 
is also immediate, and we omit the discussion of boundary cases in the following. 

2b) If t x < z > t y= o =: t, i.e., y*(t) = while x*(t) > z*(t), this however is infeasible. 

2c) If t x<z = t y=0 =: t, then y*(t + 1) > and p*(t + 1) > p*(t). Hence, x*(t + 1) < z*(t + 1), and we are 
in case (2a), which is suboptimal. 

3. If x*(0) = z*(0) and yo > 0, then this is clearly suboptimal. 
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